
Having thus described the invention, it is now claimed: 



1 . A system for organizing multi-dimensional pattern data into a reduced- 
dimension representation comprising: I 

a neural network comprised of a plurality of layers of nodes, the 

plurality of layers including: J 

an input layer comprised of a plurality of input nodes, 

a hidden /layer, and 

an outpu't layer comprised of a plurality of non-linear output 
nodes, wherein the number oy<Qli|ear output nodes is less than the number of input nodes; 

receivi/g meals for receiving multi-dimensional pattern data into the 

input layer of the neural netwo£kj_ 

output meanslfor generating an output signal for each of the output nodes 
of the output layer of the neural nitwork corresponding to received multi-dimensional pattern 

data; and I 

training melns for complying a training of the neural network, wherein 

the training means induces meai for equalizing and orthogonalWng the output signals of the 
output nodes by reducing a cov Jiance matrix of the output signals to the form of a diagona. 



matrix. 
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2. A system according to clairri 1 , wherein said training means uses 
backpropagation to iteratively update weights for the links between nodes of adjacent layers. 

3. A system according to claim 2, wherein said weights are generated 
randomly in the interval (W, -W). 

4. A system according to claim 3, wherein averaged variance of all 
dimensions of the multi-dirriensional pattern data is: 

i 

V = i|T (x. -<x>) 2 
o/7/=i p = i 



.and the elements of .he covariance maWofthe output signals of the output nodes are defined 
by: > 



I 

P 



v =±y1m -<o >){0 k -<o k >) 



.where p = l,2,...,P ; 
O is the output signal of A k.th node of the output layer for the pth input data 

k \P J 

pattern vector; ! 

O is the output signal of the k 2 th node of the output layer for the pth input data 
pattern vector; 
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<0 > is the average of O. evaluatea over the set of input data pattern vectors 
<0 > is the average of O kp evaluated over the set of input data pattern vectors 



k. = l toK ; 



k 2 =l to K ; 



K is the number of dimensions in the reduced-dimension representation; and 
< > denotes the mean evaluated q'ver the set of input data pattern vectors for each 



indicated component. 




5. A system acco/din'g to claim 4, wherein weights Aw vj between the 
hidden layer and the output layer ai i Jratively updated according to the expression: 



w . . = 



, where r\ is a constant 
oscillation; 




k-\ 



1w 



of suitabll value chosen to provide efficient convergence but to avoid 
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Ojp is the output signal from the jth node in the layer preceeding the output layer due to 
the pth input data pattern vector; 
E is the error given by: 



and, 



.where k, - k 2 - k; k = 1>~,K.; 
speed of training, 




V -r V 

out,kk kk in 



Js^positive constant which has an effect of increasing the 



.where k 2 >k,; k,= l K-l; i 2 =k l + l K; and is a positive constant which has an 

effect of increasing the speed 6f training; and 

S m 6_ , + + ©L, .where 6 to is a value proportional to the contribution to the 
error E by the outputs of thef kth node of the output layer, for the pth input data pattern vector, 
and 5^., ,5,^ , and 6^,,, are* components of 6,, 
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6. A system according t]o claim 5, wherein: 



L 



P 



J dw kj p '-> 



k dE 




i j *-i dis, . i ^_ 



where m,,, is the contribution from the diagonal terms of the covariance matrix of the outputs, 
aw v j is the contribution from the off-diagonal terms in *th row, 
A w VJ is the contributiol from the off-diagonal terms in kth column, and 
O jp is the ouput signal/from the jth node in the layer preceeding the output layer for the 

pth input data pattern vector. 
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7. A system according to claim 6, wherein: 

V, = 4 (^.ar'u^(<0 4 >-0^l-<V 



5, , = 2 



v *>■**' 



o ip d -o kp ) 



jfc-1 

E 



^„i 1 »( < ° i > -°*P 




i node in the output layer for the pth input data 



, where 

O kp is the output signal from the kt 
pattern vector, and 

< 0k > is the average of evaluated over the set of input data pattern vectors. 



8. A system according to claim 5, wherein backpropogation of error to the 
weights Aw j; between the jth node in a layer of nodes and the ith node in its' preceeding layer: 
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where, 6,. is given by: 



6. = 
jp 



w 



hi 



o JP 0 -o jp ) 



9. A method for effecting the organization of multi-dimensional pattern 
data into a reduced dimensional repientajion using a neural network having an input layer 
comprised of a plurality of input nodel^dden layer, and an output layer comprised of a 
plurality of non-linear output nodes, wherein the number of non-linear output nodes is less 
than the number of input nodes, said methjod comprising: 

iving multi-dimensional pattern data into the input layer of the neural 



receiving 



network; 



generating an output signal for each of the ouput nodes of the neural 
network corresponding to received multi-dimensional pattern data; and 

training the neural Ltwork by equalizing and orthogonal izing the output 
L : — - matrix of the output signals to the form of 



signals of the output nodes by reducing a covanance 



a diagonal matrix. 
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10. A method according to claim 9, wherein said step of training includes 
backpropagation to iteratively update weights for' links between nodes of adjacent layers. 

i 
I 

11. A method according to cl'aim 10, wherein said weights are generated 
randomly in the interval (W, -W). 



12. A method accord ing^elaim 1 1 , wherein averaged variance of all 



dimensions of the multi-dimensional partem data is: 



spt: 




,and the elements of the covariance matrix of the output signals of the output nodes is: 



v =_Vr(i) -<o >)(O b -<o.>) 

r p = \ 

.where p = l,2,...,P ; 
O is the output signal of the k.th node of the output layer for the pth input data 



pattern vector; 



O is the output signal of the k 



pattern vector; 



th node of the output layer for the pth input data 
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<n > is the average of O y evaluated over the set of input data pattern vectors 

i 

<n > is the average of O. evaluated over the set of input data pattern vectors 



k, = 1 to K ; 
k 2 =l to K ; 



K is the number of dimensions in the reduced-dimension representation; and 

< > denotes the mean evaluated over the set of input data pattern vectors for each 



indicated component. 



13. A method according/to c 



aim 12, wherein weights Aw kj between the 



hidden layer and the output layer are iteratively updated according to the expression 




Aw k . = -TV 

= -ri 
= Aw 



dE 



7W 



dE 



JP 



dE 



k-\ 



dwl. * 2 =* + i dw kj 
+ Aw,., + Aw,., 

kj,\ kj,2 kj,3 



1w 



where n is a constant of suitable value chosen to provide efficient convergence but to avoid 



oscillation; 
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O jp is the output signal from the jth node in the layer preceeding the output layer, due 
to the pth input data pattern vector; 
E is the error given by: 



a:; k 



and, 




,where k, = k 2 = k; k = 1.....K; and | kk is a positive constant which has an effect of increasing the 
speed of training, 



.where k 2 >k,; k, = l,...,K-l; k 2 =k, + l,. 



effect of increasing the speed of training; and 
6,,= 6^+6^ + 6^3 .where 6 



K- and r is a positive constant which has an 



lp is a value proportional to the contribution to the 



error 



E by the outputs of the kth node of thl output layer, for the pth input data pattern vector, 



and 6 kPil ,6 kp . 2 , and 6 kp . 3 are components ot 



6 
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14. A method according to claim 13, wherein: 




- ^ kp , 2 o jp 



p = l 



k\-\ be. . i p 

*',=i dw kJ Pp-X 



where aw Vi , is the contribution from the diagonal term, 



is the contribution frorr 



the off-diagonal terms in kth row, and 



is the contribution from the off-diagonal terms in kih column. 



by: 



15. A method according to claim 14, wherein 6^,,, 6 Ap . 2 and 6^ are given 



where 



kp,i 



Ev \ (<o>-o. ) 



/ 



*p,3 



Jfe-1 




O kp is the output signal from the 



kth node in the layer preceeding the output layer for the 



pth input data pattern vector, and 

<0^> is the average of evaluated over the set of input data pattern vectors. 

,6. A method according to claim 13, wherein backpropogation of error to 
the weights AWjj between the jth node |„ a Uyer of nodes and the i.h node in its' preceeding 
layer are: 
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v \ji 



where, 6:. is given by: 



6. = 

JP 



17. A system for organizing multi-dimensional pattern data into a reduced 

dimensional representation comprising: 

a neural network comprised of a plurality of layers of nodes, the 

plurality of layers including: 

input layeWoiiprised of a plurality of input nodes, and 
nput layer cLprised of a plurality of non-linear output 
nodes, wherein the number of non-linear output nodes is less than the number of input nodes; 

receiving means for receiving multi-dimensional pattern data into the 

input layer of the neural network; 

output means for generating an output signal at the output layer of the 

neural network corresponding to receive 



an 



an out 



ed multi-d^nensional pattern data; and 
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the training means conserves a measure 



training means for completing a training of the neural network, wherein 
of the total variance of the output nodes, wherein the 



\ 



total variance of the output nodes is defined as: '; 

\ 



V={\IP)Y,X{x ip -<x>) 7 
p = l i=l 




, where {x p }is a set of data pattern vectors; 
p=l,2 P; 

P is defined as a positive integer; 
< x, > denotes the mean value of of x ip evaluated over the set of data pattern 

vectors; 

S is the number of dimensions; 
x ip is the ith component of x p , the pth member of a set of data pattern vectors. 

18. A system according to claim 17, wherein said training means completes 
the training of the neural network via backpropagation for progressively changing weights for 
the output nodes. «. 
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includes, 



19. A system according to din 18, wherein said training means further 



means for training the neural network by backpropagation by 
progressively changing weights w kj at the output layer of the neural 

:hl 



network in accordance with'. 



Aw,.=(l/P)2^ Aw 
P =i 



kj =^IP)T,^ b P k° P j 



signal from the jth nli/jU layer preceeding the output layer 



due 



, where O pj is the output 
to the pth data pattern, 

of suitable value chosen to provide efficient convergence but to avoid 



rj is a constant 
oscillation, and 

6 is a value proportional to the contribution to the error E by the outputs of the kth 
node of the output layer for the pth input data pattern 



20. A system according to claim 



q n 



9, wherein: 



oy)(.o.-<o k >)o ft (\-o pk ) 



\ 
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21 . A system according to claim 19, wherein said neural network rurtner 
comprises at least one hidden layer comprised of hidden nodes, wherein adaptive weights w, 
for each hidden node is progressively improved in accordance with, 



where 



O pi is the output signal for the ith node of the layer preceeding the jth layer of the pth 



input data pattern. 




22. A system accordi|i^E^raim 21, wherein: 



/ 



6 = 
PJ 



K 



P k 



o p p-o p p 



23. A method for effecting the organization of multi-dimensional pattern 
data into a reduced dimensional representation using a neural network having an input layer 
comprised of a plurality of input nodes, and an output layer comprised of a plurality of non- 
linear-output nodes, wherein the number o| non-linear output nodes are less than the number 
of input nodes, said method comprising: V 
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receiving a set {x p } of data pattern vectors into the input layer ot tne 
neural network, wherein p=l,2,...,P and wLein P is defined as a positive integer, and 
wherein the set of data pattern vectors has a 



P =p 



total variance defined as, 



1 = 1 



,\vhere {x p } is a set of data pattern vector^ 
p= 1,2,...,P; 

P is defined as a positive integer; 
< Xi > denotes the mean value of cjf x ip evaluated over the set of data pattern vectors; 

S is the number of dimensions; 

x- is the ith component of x pf the pith member of a set of data pattern vectors; 




training the neural 
displaying a multi- 



nfctwork by backpropagation; and 
dimensional output signal from the output layer of the 



neural network. 
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24. A method according to claim 23, wherein said step of training the neural 
network by backpropogation includes progrep sively changing weights w kj at the output layer of 
the neural network in accordance with, 



Aw v -(l/P)X) 



a^ki/^E^o,; 

P = l 



, where O pj is the output signal from the jth node in the layer preceeding the output layer due 
to the pth data pattern, and 

r| is a constant of suitable valure chosfc 



en to provide efficient convergence but to avoid 



oscillation. 

6 pk is a value proportional to the contfibution to the error E by the outputs of the kth 
node of the output layer for the pth input datja pattern. 

25. A system according tolclaim 24, wherein: 



s^k-o/^E E (o\ n -<oy](o pt -<o k >)o pk v -o pk ) 

q n 
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4 



* 



26. A method according to c 
comprises at least one hidden layer comprised 

for each hidden node of the neural network is progressively improved in accordance with 



aim 23, wherein said neural network further 
f hidden nodes, wherein adaptive weights w 



Aw.. = -T| 



, where O pi is the output signal for the ith node 
input data pattern. A 



27. A method according 



toe 



„ I pi pi 



P =\ 



of the layer preceeding the jth layer of the pth 



6 .= 
pj 



aim 26, wherein 



P k 



28. A method according to c 
signal is a two-dimensional output signal. 



aim 23, wherein said multi-dimensional output 



29. A method according to claim 23, wherein said two-dimensional output 




ignal includes data points plotting in relation to 2-dimensional axes. 



-63- 



